How to Read a Quantitative Paper

Application: How much counties shifted to Trump

Know the variables and what they mean

• Each observation/data point is one county
• Shift to Trump: How much more did the county vote for Trump in 2016 than it voted for Romney in 2012
• Median income ($1,000s): The county median income, expressed in units of $1,000. That is, a county with a $50,000 median income will be coded as 50. This is done to improve your ability to read the regression results.

Regression function form

Recall that

• \(\beta\) is the slope, also called a coefficient
• c is the intercept

and the equation can be expressed as: \[\begin{align} y &= \beta x + c \\ \text{shift to Trump} & = \beta * \text{median income (\$1,000s)} + c. \end{align}\]

Preview: How to Scan Tables

• Step 0: Know what the variables mean
• Step 1: For a first understanding, pay attention to coefficients/slopes with stars (this means the value has a statistical significance threshold with p < 0.05, which roughly speaking means we believe the value is not due to chance with 95% confidence). Ignore the constant term regardless. Take another stats class to know why the stars are very approximate indicators of substantive meaning.
• Step 2: Look at the direction of the coefficient/slope. Is it positive or negative?
• Step 3: ONLY for linear regressions/OLS: a one unit change in the x variable is associated with a \(\beta\) unit change in the y variable.

	% shift to Trump, 2012-2016
median income (1,000s)	-0.158***
	(0.006)
Constant	8.337***
	(0.351)
Observations	3,111
Adjusted R2	0.203
Note:	p<0.05; p<0.01; p<0.001

Step 1: The coefficient/slope for county income has stars

• Therefore we believe, with at least 95% confidence, that the sign of the coefficient is real.
• You can roughly calculate whether the coefficient/slope will have stars by using the number in parentheses below the coefficient/slope, which is the standard error, a measure of confidence in the coefficient. If the 2*(standard error) < |slope|, then there are likely to be stars on the coefficient/slope.

Step 2: The coefficient/slope for county income is negative

• There is a negative relationship between income and shifting to Trump. Because it has stars, we think, with 95% confidence, this negative sign is not due to chance alone.
• This value of the coefficient, -0.158, is called the estimated effect size of county median income on shifting to Trump between 2012 and 2016.

Step 3: A one unit change in county median income is associated…

• A $1,000 increase in county median income is associated with a -0.158% shift to Trump between 2012 and 2016.
• Looking at the scatterplot, the range of the values for county median income is from roughly $25,000 to $125,000. That’s 100 units. So a county at the bottom end of the spectrum has shifts to Trump by around 15% more than a county at the highest income levels. This is a substantial change.
• Note: a significant/meaningful change in the outcome does NOT mean the slope/coefficient is statistically significant. Statistically significant and substantively significant are very different concepts.

Multiple regression: adding variables

Know the variables and what they mean

• Each observation/data point is one county

• Shift to Trump: How much more did the county vote for Trump in 2016 than it voted for Romney in 2012

• median income ($1,000s): The county median income, expressed in units of $1,000. That is, a county with a $50,000 median income will be coded as 50. This is done to improve your ability to read the regression results.

• % college experience: The percent of the county with at least some college experience, regardless of whether they earned a degree.

Regression function form

Recall that

• \(\beta\) is the slope, also called a coefficient
• c is the intercept

and the equation can be expressed as: \[\begin{align} y &= \beta_1 x_1 + \beta_2 x_2 +c \\ \text{shift to Trump} & = \beta_1 * \text{median income (\$1,000s)} + \beta_2 * \text{% college experience} +c . \end{align}\]

	% shift to Trump, 2012-2016
	(1)	(2)
median income (1,000s)	-0.158***	-0.013
	(0.006)	(0.007)
% college experience		-0.344***
		(0.012)
Constant	8.337***	20.103***
	(0.351)	(0.512)
Observations	3,111	3,111
Adjusted R2	0.203	0.371
Note:	p<0.05; p<0.01; p<0.001

See https://drellaswat.github.io/trump_incomeedu_plane/ for a bigger version of this plot.

Step 1: The coefficients for county income AND county education have stars

• Therefore we believe, with at least 95% confidence, that the signs of the coefficients are not due to chance.

Step 2: The coefficients for county income AND county education are negative

• Adding in new variables, like education, controls for education. This means that each coefficient shows how much the outcome, shifting to Trump, changes as that variable changes, holding all other variables constant. This is called the marginal effect.

• Example: A one unit change in county median income is associated with a -0.013 change in shifting to Trump when county education is held constant. You could imagine only looking at counties where around 40% of the county had attended college. Among those counties, a county with a median income of $50,000 is expected to shift to Trump 0.013% less than a county with a median income of $51,000. The same would be expected if we isolated to only counties with around 60% education.

• Notice that the effect of income is much smaller when education is added to the equation.

• Adding a new control/additional variable to a regression can change a coefficient’s value substantially when the new variable is strongly correlated with the original variable.

Step 3: A one unit change in x is associated…

County income

• A $1,000 increase in county median income is associated with a -0.158% shift to Trump between 2012 and 2016.
• Looking at the scatterplot, the range of the values for county median income is from roughly $25,000 to $125,000. That’s 100 units. So a county at the bottom end of the spectrum has shifts to Trump by around 1.3% more than a county at the highest income levels. This is… not a substantial change.
• Note: The coefficient/slope is statistically significant; we believe it is truly negative. My read is that it is NOT substantively significant; the size of the change is not very important. Statistically significant and substantively significant are very different concepts.

County education

• A 1% increase in county college experience is associated with a -0.344% shift to Trump between 2012 and 2016.
• Looking at the scatterplot, the range of the values for county median income is from roughly 20% to 90%. That’s 70 units. So a county at the bottom end of the spectrum has shifts to Trump by around 24% more than a county at the highest income levels. This is a substantial change.
• Note: In this case, education is both statistically and substantively significant.